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PEN-INPUT DEVICE 
FIELD OF INVENTION 

The present invention relates generally to pen input devices, in particular to pen input 
devices using a two-dimensional camera for determining movement of the pen. 
5 BACKGROUND OF THE INVENTION 

Personal computers (PC) have become almost indispensable. As such, flexibility in 
entering data to them is important. In addition to PCs there are many other types of electronic 
devices which require simple and/or flexible data entry, for example, PDAs, cell phones, 
watches, pocket dictionaries, set-top boxes, network computers, writing boards and remote 
10 controls. 

Gyroscopic handwriting recognition is known. British patent GB 2,306,669, "Manual 
Entry Interactive Paper and Electronic Document Handling and Processing System", the 
disclosure of which is incorporated herein by reference, utilizes one or more gyroscopes to 
determine motions of a pen and recognize handwriting. The pen input device also includes a 

15 camera for optical recognition of pre-printed bar codes. 

Optical recognition of printed matter is known. US patent 4,538,072, "Optical Wand for 
Reading OCR Characters and Bar-Code Marks'*, the disclosure of which is incorporated herein 
by reference, describes an optical wand which has a camera for scanning bar codes and 
characters, for OCR (optical character recognition). 

20 Handwriting recognition on a girded surface, is known. US patent 5,420,943, 

"Universal Computer Input Device", the disclosure of which is incorporated herein by 
reference, describes an input device which can be used as a bar-code scanner, a mouse, a text 
scanner or a handwriting input device. A linear CCD array is used to determine movement of 
the pen tip relative to a specialized printed grid. 

25 US patent 4,73 1 ,526, "Process for Picking -up Navigation Data", the disclosure of which 

is incorporated herein by reference, describes a process for determining relative movement on a 
map using a photocell to detect changes in reflection from a grid overlay and/or a printed grid. 
A resistive type overlay is suggested for determining absolute positioning. This process is used 
for scanning but not for writing. 

30 US patent 4,906,843 "Combination Mouse, Optical Scanner and Digitizer Puck", the 

disclosure of which is incorporated herein by reference, describes a combination mouse, 
digitizer puck and scanner which can operate as a scanner, a mouse or a digitizer puck. The 
puck utilizes flux detectors and the mouse utilizes optical movement encoders. 
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SUMMARY OF THE INVENTION 

One aspect of some preferred embodiments of the invention relates to determining a 
motion of a pen tip utilizing a two dimensional camera which images a vicinity of the tip. 
Preferably, movements of the tip are used to recognize handwriting inputted by the pen. In a 

5 preferred embodiment of the invention, the motion determination compares successive images 
acquired by the camera to determine motion. The motion determined may include position 
and/or orientation, and/or first and/or second derivatives thereof (i.e., pen velocity and pen 
acceleration). Preferably the images are compared by correlation. Alternatively or additionally, 
the motion determination utilizes an image of a trace marked by the pen itself as it moves on an 

10 imaged writing surface. Thus, while a gridded or even an arbitrarily patterned writing surface 
may be used, movements of the pen tip may be detected even on an (previously) unmarked 
writing surface. In a preferred embodiment of the invention, the position is a two dimensional 
orientation. However, in some embodiments, only a single dimension is required and/or 
determined. Additionally, even though three orientation angles are available (rotation, 

15 horizontal slant and vertical slant), any number may be determined by the pen, including zero 
and one. Alternatively or additionally, a composite pen slant may be determined. 

In a preferred embodiment of the invention, the pen includes a pressure transducer to 
detect when the pen is pressed against a writing surface and/or an amount of applied pressure 
and/or that the pressure is over a threshold. Preferably, the contact determination is used to 

20 decide if writing is to be detected. Alternatively or additionally, the relative positions of the pen 
and surface and/or relative movements thereof are determined using a range finder or by optical 
processing of the image acquired by the camera. 

An aspect of some preferred embodiments of the invention relates to identification of an 
absolute position of a pen. Two types of absolute positioning may be identified. In a first type 

25 of positioning, an area in which the pen is positioned is identified. In one example, a pre- 
printed form may include a plurality of areas, each with a different background pattern. When 
the pen is positioned in one of the areas, the background pattern of the area may be analyzed by 
the pen to determine the identification of the area. A second type of absolute positioning 
optically encodes a position in a grid. Thus, an image of a grid portion underlying the pen may 

30 be analyzed to determine an absolute position and/or orientation of the pen relative to the grid. 

An aspect of some preferred embodiments of the invention relates to using the same pen 
device for handwriting recognition, scanning of bitmap images and/or inputting of graphical 
data, by tracing. Alternatively or additionally, the pen may be used as a pointing device (e.g. 
emulating a mouse). In a preferred embodiment of the invention, a pre-printed pad is provided 
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with differently patterned areas for a mouse pad, a writing area and/or function keys. When the 
pen is positioned over the mouse pad, it operates as a mouse, when it is positioned over the 
writing surface, it functions as a handwriting input device and when positioned over a function 
key, the pen activates the key, as if pressed on a keyboard. 

In a preferred embodiment of the invention, the grid is invisible to a human eye, for 
example being reflective mainly in the infrared spectrum and/or it being printed using a 
fluorescent material which is activated by a special light source on the pen. In a preferred 
embodiment of the invention, the grid is provided by overlaying a transparency having a grid 
printed thereon, over a work surface. 

An aspect of some preferred embodiments of the invention relates to using a pen device 
to annotate printed text. In one example, the text comprises a form, possibly partially filled, and 
a user enters text and/or corrections to existing text using a pen input device. In another 
example, the text comprises a printed page. 

An aspect of some preferred embodiments of the invention relates to controlling the 
functionality of a pen input device. Preferably, various functions of the pen and/or of a software 
application controlling the pen are controlled and/or initiated using switches on the pen. 
Alternatively or additionally, the pen recognizes gestures and/or written symbols made by the 
pen as indicating specific function requests. Such function requests may include control of the 
pen itself, entering non-printable characters (such as "delete") and/or indications of actions to 
perform on a text, for example to highlight a circled text section. 

An aspect of some preferred embodiments of the invention relates to handwriting 
recognition on multiple surfaces, including for example, paper, computer screens (in which a 
pattern may be actively displayed and/or may be passively formed by the display elements 
themselves), black-board, white-board and/or pre-printed forms. In a preferred embodiment of 
the invention, the pen device is provided as a camera attachment to an existing writing 
instrument, for example a pen or a marker. 

An aspect of some preferred embodiments of the invention relates to electronically 
controlling the flow of ink in a pen (or engraving action in an engraving tool). Preferably, ink is 
allowed to flow only if writing is allowed, for example on a white sheet of paper. Preferably, 
the allowance of writing on a surface is responsive to an identification of a pattern printed on 
the writing surface. 

An aspect of some preferred embodiments of the invention relates to various levels of 
connection between a pen device and a target computer. Although the pen may be connected to 
a computer by wire, in a preferred embodiment of the invention, the connection is a wireless 
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connection. The processing ability of the pen may be provided on the pen itself, on a connected 
computer or utilizing a dedicated processing hardware, preferably connected between the pen 
and the computer. 

In a preferred embodiment of the invention, the pen includes an internal memory and/or 
5 processor for storing handwriting and/or commands. Thus, the pen may be used as a stand 
alone device for marking up a printed copy of a document. When the pen is connected to a 
computer, an electronic copy of the document is updated and, optionally printed. Alternatively 
or additionally, the pen may comprise a computer, with an external connection required to print 
out a document. 

10 There is thus provided in accordance with a preferred embodiment of the invention a 

method for determining a motion of a tip on a surface having a pattern of distinct features 
marked thereon, comprising: 

imaging the pattern at least in the vicinity of the tip, using a two-dimensional camera 
coupled to the tip; 
15 comparing successive images acquired by the camera; and 

analyzing the images to determine the motion of the tip. 

Preferably, the method comprises forming at least a portion of said pattern by ink 
released from said tip. Alternatively or additionally, comparing comprises correlating at least 
parts of the images. Alternatively or additionally, comparing comprises feature extraction and 

20 matching of said images. 

Alternatively or additionally, the method comprises providing said pattern on the 
surface prior to the imaging, wherein said pattern includes a coded pattern for absolute 
positioning of the tip. Preferably, the coded pattern comprises a plurality of areas, each with a 
different background pattern and said camera has a field of view larger than an area of a single 

25 pattern. Preferably, the plurality of areas comprises a plurality of square bar codes that form a 
checkerboard. 

Alternatively or additionally, the coded pattern is a coded grid, wherein each line of the 
grid is a specific combination of lines of varied thickness and spaces of different widths. 
Preferably, the coded grid has a likeness of a woven lattice. 
30 In a preferred embodiment of the invention, at least part of said pattern is invisible to a 

human eye. Preferably, said pattern is reflective in the infrared spectrum. Alternatively or 
additionally, said pattern comprises a fluorescent material. Alternatively or additionally, said 
surface comprises a transparency having said pattern printed thereon. 
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Alternatively, said surface comprises a plain paper with a pattern printed thereon. 
Alternatively or additionally, said pattern comprises a grain of the surface. 

In a preferred embodiment of the invention, determining the motion of the tip includes 
determining relative translation. Alternatively or additionally, determining the motion of the tip 
includes determining relative orientations. Alternatively or additionally, determining the 
motion of the tip includes determining an absolute position. Preferably, said position comprises 
a one-dimensional position. 

Alternatively or additionally, said position comprises a two-dimensional position. 

In a preferred embodiment of the invention, determining the motion of the tip includes 
determining an absolute orientation. 

In a preferred embodiment of the invention, said orientation comprises a rotation angle 
of said tip around an axis perpendicular to the surface. Alternatively or additionally, said 
orientation comprises an angle between said tip and said surface. 

In a preferred embodiment of the invention, determining the motion comprises 
determining a velocity of the motion. Alternatively or additionally, determining the motion 
comprises determining a rate of change of the velocity of the motion. 

In a preferred embodiment of the invention, the method comprises utilizing said 
detected motions to recognize handwriting. Preferably, handwriting recognition comprises 
handwriting recognition utilizing motion vectors. Alternatively, handwriting recognition 
comprises utilizing both motion vectors and images of the surface for handwriting recognition. 

Alternatively or additionally, handwriting recognition comprises: 

generating an image utilizing said determined motion; and 

recognizing at least one symbol in said generated image. 

Alternatively or additionally, handwriting recognition comprises utilizing a user- 
dependent database to perform said handwriting recognition. 

In a preferred embodiment of the invention, the method comprises utilizing said 
determined motions for graphical input. Alternatively or additionally, the method comprises 
utilizing said determined motion for controlling- a cursor. Preferably, controlling a cursor 
comprises emulating a mouse. 

In a preferred embodiment of the invention, the method comprises reconstructing a 
scanned image from said acquired images, utilizing said determined motions. Preferably, the 
method comprises analyzing said images to recognize special codes. 

In a preferred embodiment of the invention, said tip forms a part of a pen input device, 
and the method comprises: 
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recognizing a particular pattern in at least one of said acquired images; and 
modifying an operational mode of said pen input device responsive to said recognized 
particular pattern. 

Alternatively or additionally, said tip forms a part of a pen input device, and the method 
5 comprises: 

receiving a voice input by said pen; and 

modifying an operational mode of said pen input device responsive to said voice input. 
Alternatively or additionally, said tip forms a part of a pen input device, and the method 
comprises: 

10 recognizing an entry of a symbol utilizing said determined motions; and 

modifying an operational mode of said pen input device responsive to said recognized 
symbol. 

Alternatively or additionally, said tip forms a part of a pen input device, and the method 
comprises: 

15 recognizing said determined motions as gestures; and 

modifying an operational mode of said pen input device responsive to said recognized 
gestures. 

Alternatively or additionally, said tip forms a part of a pen input device, and the method 
comprises: 

20 detecting when the tip in contact with said writing surface; and 

modifying an operational mode of said pen input device responsive to said detection. 
Preferably, said modifying comprises detecting motion of the tip responsive to said 
detected contact. Alternatively or additionally, said modifying comprises operating said camera 
responsive to said detected contact. 
25 In a preferred embodiment of the invention, modifying comprises selectively operating said 
pen input device in at least one of the following input modes: handwriting input, graphical 
input and pointer emulation. Preferably, the input modes include a hypertext link selection 
mode, in which the pen may be selectively operated. 

In a preferred embodiment of the invention, modifying comprises controlling said 
30 camera. Alternatively or additionally, modifying comprises controlling the release of ink. 

There is also provided in accordance with a preferred embodiment of the invention, a 
method of controlling ink flow in a pen, comprising: 

determining when said pen is in contact with a surface whose marking is allowed; and 
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automatically electronically controlling a flow of ink, responsive to said determination. 
Preferably, determining comprises determining if said pen is operating in a pointer emulation 
mode. 

There is also provided in accordance with a preferred embodiment of the invention, a 
5 pen input device comprising: 
a tip; 

a two dimensional camera, coupled to said tip, which images a surface at the vicinity of 
the tip; and 

circuitry for determining motion of said tip relative to said surface, by analysis of 
10 successively acquired images. 

Preferably, the pen comprises an electrical wire connection to a computer. 
Alternatively, the pen comprises a wireless connection to a computer for data generated in the 
pen. 

In a preferred embodiment of the invention, at least some of said circuitry is comprises 
15 in a dedicated processing unit. Alternatively or additionally, at least some of said circuitry 
comprises software operable with on said computer. 

In a preferred embodiment of the invention, said pen comprises a memory. Preferably, 
said memory stores at least portions of said images. Alternatively or additionally, said memory 
stores data responsive to said movement determination. 
20 In a preferred embodiment of the invention, said pen comprises an internal power 

source. Alternatively or additionally, the pen comprises a light source, coupled to the pen input 
device, to illuminate the surface. Alternatively or additionally, the pen comprises a pressure 
transducer which determines if the tip is in contact with the surface. Alternatively or 
additionally, the pen input device is selectively operable in a plurality of input modes, 
25 including at least handwriting recognition and pointer emulation. Preferably, the input modes 
include pattern recognition. Alternatively or additionally, the input modes include hypertext 
link selection. 

In a preferred embodiment of the invention, the pen comprises an external software, and 
said mode is determined by the software. Preferably, said software controls the pen to operate 
30 in said different modes. Alternatively or additionally, said software analyses signals from the 
pen, responsive to said mode. 

In a preferred embodiment of the invention, the pen comprises a personal computer on 
which said software is executed. 
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In a preferred embodiment of the invention, the pen comprises at least one switch for 
controlling the operation of the pen. Alternatively or additionally, said tip comprises an ink 
dispenser. Preferably, the pen comprises an ink flow control which selectively allows ink to be 
dispensed from said tip onto said surface. 

In a preferred embodiment of the invention, said circuitry comprises a memory for 
reconstructing virtual pen marking from said determined movements. 

Alternatively or additionally, said circuitry recognizes handwriting patterns based on 
said determined movements. 

Alternatively or additionally, said circuitry determine a three-dimensional position of 
said tip relative to said surface. 

Alternatively or additionally, said circuitry determines a three-dimensional orientation 
of said tip relative to said surface. 

Alternatively or additionally, said pen comprises a rechargeable power source. 

There is also provided in accordance with a preferred embodiment of the invention, an 
implement mountable camera, comprising: 

a camera having a field of view; and 

a bracket for attaching said camera to a writing implement, such that said field of view 
images at least a vicinity of a tip of said elongate writing implement. 

Preferably, the camera comprises circuitry for analyzing images acquired by said 
camera to determine motion of said tip. 

There is also provided in accordance with a preferred embodiment of the invention, a 
controllable pen having a pen tip, comprising: 

an ink dispenser; and 

an ink flow control which selectively blocks ink from being dispensed from said ink 
dispenser. Preferably, said ink flow controller comprises an electrical flow controller. 

There is also provided in accordance with a preferred embodiment of the invention, a 
pen input device comprising: 

a tip; 

a two dimensional camera coupled to said tip and which images a surface at the vicinity 
of the tip; and 

a rechargeable power source for powering said camera. Preferably, the pen comprises a 
connector to a computer, wherein said connector is adapted to hold said pen while not is use 
and wherein said connector comprises a data communication adapter for said computer and a 
power adapter to recharge said rechargeable power source. 

8 



WO 00/25293 



PCT/US98/22477 



BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will be more clearly understood with reference to the following detailed 
descriptions of non-limiting preferred embodiments of the invention in which same reference 
numbers are used to denote same articles throughout the figures and in which: 
5 Fig. 1 is a schematic representation of a pen input device in accordance with a preferred 

embodiments of the present invention; 

Fig. 2 is a schematic representation of an alternative embodiment of a pen input device; 

Figs. 3A - 3E illustrate a handwriting recognition method in accordance with a 
preferred embodiment of the invention; 
10 Fig. 4 is a schematic representation of a data entry pad, in accordance with a preferred 

embodiment of the present invention; 

Figs. 5A-5F comprise a flowchart of a program for controlling a pen input device, in 
accordance with a preferred embodiment of the invention; 

Fig. 6 is a schematic illustration of a system configuration including a computer and a 
15 pen input device, in accordance with a preferred embodiment of the invention; 

Fig. 7A and 7B are block diagrams of a wired pen input device in accordance with a 
preferred embodiment of the invention; and 

Fig. 8A and 8B are block diagrams of a wireless pen input device in accordance with a 
preferred embodiment of the invention. 
20 DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Fig. 1 is a schematic representation of a pen input device 20 in accordance with a 
preferred embodiments of the present invention. Pen 20 includes a body 28, an ink dispenser 30 
and a camera 22 which images an area 24 of a writing surface 21, adjacent the tip of ink 
dispenser 30, preferably using a lens 26. Images acquired by camera 22 are preferably analyzed 
25 (as described below) to determine movements ofpen 20. Preferably, the signals are sent to a 
processor (not shown in this Fig.), via a connector 38. In some preferred embodiments of the 
invention, the connector is a wired connector. Alternatively or additionally, the connector 
comprises a wireless connection. 

In a preferred embodiment of the invention, pen 20 includes a pressure sensor and/or 
30 switch 36 which generates a signal responsive to the pen being put to. paper. In one 
embodiment, sensor 36 is coupled to ink dispenser 30 using a spring 34. Alternatively or 
additionally, pen 20 includes a sensor (not shown) which generates a signal responsive to ink 
dispenser 30 being in a writing position. 
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In Fig. 1, camera 22 is shown mounted on a bracket 32 external top pen 20. In some 
embodiments, camera 22 and bracket 32 are provided as a stand-alone device which may be 
attached to many writing implements, for example a favorite pen. Fig. 2 is a schematic 
representation of an alternative embodiment of a pen input device, 40, in which a camera 42 is 

5 located inside the body of pen 40. A lens 46 is preferably arranged at a writing tip of the pen, to 
image area 24. Alternatively or additionally, the camera may be provided as a wrap around for 
an existing pen, so that when the camera is mounted on the pen, an imaging configuration 
similar to that of Fig. 2 is created. 

It should be appreciated that the term "pen 11 is used herein for convenience. As used 

10 herein, a pen may be any of a plurality of writing devices known in the art, for example, a pen, 
a chalk stick, a marker, an engraving tool and/or a paintbrush. Additionally, as described 
below, pen 20, in some embodiments thereof, may be used to generate signals which are 
substantially the same as those generated by a gyroscopic pen input device or by other types of 
pen input devices. Thus, pen 20 may be used in their place in many applications. Alternatively 

15 or additionally, pen 20 may use a surface of a display, such as an LCD or a CRT as a writing 
surface. It should be noted that generating a pattern on a display is straight forward and, in 
some ways, more flexible than using a pre-printed sheet (described below). Also, many display 
devices have a noticeable grain which may server as a pattern. Furthermore, in some preferred 
embodiments of the invention pen 20 selectively leaves no mark on the writing surface. 

20 Lens 26 (Fig. 1) or lens 46 (Fig. 2) are preferably regular imaging lenses. Alternatively 

or additionally, a fish-eye lens is used, so that better resolution is achieved at portions of area 
24 nearer the tip of pen 20 and/or while imaging a larger area 24. Alternatively or additionally, 
lens 46 (or lens 26) includes a fiber optic wave guide to channel light to camera 42. In a 
preferred embodiment of the invention, pen 20 (or pen 40) includes a light source (not shown) 

25 to illuminate area 24. It should be noted that for many applications a low resolution camera, a 
camera with a wide dispersion of gain for light detectors and/or a camera in which some of the 
light detectors are damaged may be used. Preferably, the camera quality and/or resolution are 
selected to match a task for which the pen is used. 

Alternatively or additionally, lens 26 is a magnifying lens. In a preferred embodiment 

30 of the invention, the magnification is set so that a grain of a writing surface will be visible. For 
example, a hand and/or a table will appear to have a pattern marked thereon if the 
magnification is sufficient. 

In a preferred embodiment of the invention, lens 26 comprises a zoom lens, having 
variable magnification settings (and/or focus settings). Preferably the lens is a manual lens. 

10 
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Alternatively or additionally, lens 26 comprises an electronic zoom lens. In a preferred 
embodiment of the invention, the zoom setting is set during a calibration and/or responsive to 
the task to be performed. Alternatively or additionally, the zoom setting is automatically 
determined based on an imaged grain size of a writing surface. Thus, in some cases, the zoom 
setting may change during a single task. 

In a preferred embodiment of the invention, movements of pen 20 are determined by 
comparing consecutive images acquired of areas 24 and analyzing the images to determine a 
relative movement and/or a relative rotation between the images. In a preferred embodiment of 
the invention, the images are compared by correlation of, at least parts of, the images. 
Alternatively or additionally, the images are compared using other techniques known in the art, 
for example feature extraction and then feature matching. 

In some preferred embodiments of the invention, writing surface 21 includes a grid 
marking, possibly in which each grid line comprises a plurality of lines. The number, thickness 
and/or distance between the lines may be used to determine a relative and/or absolute position. 
In a preferred embodiment of the invention, the lines in the horizontal and vertical directions 
are overlaid, to form a weave-like pattern. Alternatively or additionally, the lines simply meet 
at vertexes of the grid. Alternatively or additionally, the marking comprises a checkerboard of 
bar-codes. Alternatively or additionally, the marking comprises an arbitrary, possibly non- 
repeating pattern. Thus, images of two overlapping areas 24 are more easily correlated to 
determine movement. However, it should be appreciated that such marking is not necessary in 
all preferred embodiments of the invention. Rather, the ink dispensed by ink dispenser 30 can 
provide enough information content (possibly by itself) in areas 24 for a successful movement 
detection. 

In a preferred embodiment of the invention, sensor 36 is used to determine if pen 20 is 
in contact with writing surface 21 . Preferably, movement detection is performed only when pen 
20 is in contact with surface 21. Alternatively or additionally, when pen 20 is not in contact 
with surface 21, pen 20 works in a different mode. In one example, a focal length of lens 26 
changes so that camera 22 images objects further away, such as a nearby writing surface. In 
another example, pen 20 changes to a volume motion mode, in which a three-dimensional 
position is determined, for example by measuring a three-dimensional vector of motion flow. 
An image of surroundings acquired by camera 22 may be analyzed to determine motion in up 
to six degrees of freedom: rotation of the image, scale of the image, X and Y axis movement 
and distortion along X and along Y axes. Thus, spatial gestures made with the pen may also be 
captured. 

11 
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Alternatively or additionally, sensor 36 may be used to signal other changes in the 
mode of operation of pen 20, for example, operating in a manner analogous to a mouse button 
and/or a pressure pad. 

Handwriting recognition may then be performed utilizing the acquired images and/or 

5 the determined movements. Two basic types of handwriting recognition may be applied, vector 
based and bit-mapped based. In the bit-mapped based handwriting recognition method, 
individual images of areas 24 are accumulated into a larger bit map, using the correlation 
and/or the movement analysis. An OCR algorithm, many of which are known in the art, may 
then be applied to the large bit-map, to determine the character. The pen may also buffer the 

10 larger bit map, so that entire words are OCRed in a single operation. 

In the vector-based handwriting recognition method, the movement vectors themselves 
are used to identify the letter. In addition, sensor 36 and/or an analysis of the acquired images 
may be used to determine when pen 20 did and/or did not contact surface 21. The frame rate of 
the acquired images (and movement analysis) is preferably determined to match an expected 

15 pen motion speed, letter complexity (which is different for English and Chinese) and/or 
geometry of area 24. Preferably, a desired frame rate is between 10 and 24 frames per second. 
It should be noted however that the snapshot time for an image may be very short, for example 
one millisecond. Alternatively or additionally, the snapshot time is longer and the acquired 
images are analyzed to detect motion based on a direction and/or magnitude of smear in the 

20 image. 

Alternatively or additionally, a combination of methods may be used, for example using 
vector information as hints for the OCR process or using an image of the actual motion, in 
association with the motion vector, to better define the exact trajectory of the pen tip. 

Figs. 3A-3E illustrate a handwriting recognition method in accordance with a preferred 
25 embodiment of the invention. In Figs. 3A-3D, pen 20 generates consecutive line segments 50a- 
50d and camera 22 acquires consecutive images 52a-52d. Each pair of consecutive images 
(e.g., 52a,52b) are analyzed to determine relative motion of the tip of pen 20. Alternatively or 
additionally, non-consecutive images may also be analyzed to determine relative motion 
therebetween. Non-consecutive image comparison is especially useful to correct for drift errors 
30 which may accumulate when serially comparing a large number of images. 

Fig. 3E illustrates an exemplary application of the present invention, namely, a word 
processor 60. An area 64 is a main display area and includes a cursor 66, to the right of which a 
new character is to be entered. A handwriting recognition area 62 may be used to display a 
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character as it is being entered. Once the character entry is complete, the character is moved to 
the next character location to the right of cursor 66. 

Alternatively or additionally, pen 20 may be used for Internet browsing. In a preferred 
embodiment of the invention, the pen is used to select hypertext links, for example by 
indicating a link on the screen (e.g., circling them) and/or by scanning a link. In a preferred 
embodiment of the invention, the image of the link is within the field of view of the camera, 
when the link is circled. Alternatively or additionally, the word, in the link is written out to 
activate the link. Alternatively or additionally, the camera image is used to identify patterns, for 
example those which indicate specific links. 

Alternatively or additionally, pen 20 is used for data entry in preparing of e-mail, to 
select and7or enter a telephone number to be dialed, as part of a chat session and/or for multi- 
user games, on a network or on a single computer (with multiple pens connected thereto). It 
should be noted that these applications may thus be used without requiring a keyboard, in 
accordance with some preferred embodiments of the invention. 

Alternatively or additionally, the data entry using pen 20 may be used for handwriting 
analysis, for example for signature verification, for example for secure electronic commerce. 
Alternatively or additionally, the handwriting analysis may be used for graphology and/or for 
education, e.g., for teaching of handwriting. 

Area 62 is not required in all embodiments of the invention, since, in some 
embodiments, any surface may be used for data entry and an ink feedback is provided to the 
user regarding his actual pen motions.. Thus, display 62 may be a low resolution display, for 
example a low resolution numeric display. Such a low resolution display is usually cheaper, 
lighter and/or requires less power to operate than a high-resolution graphic display. 

Alternatively or additionally, to using sensor 36 to determine contact of pen 20 with 
surface 21, other technologies may be used. In one example, a quality of focusing of the 
acquired images is used to determine a distance between camera 22 (and hence pen 20) and 
surface 21. Alternatively or additionally, a range finder, such as an ultrasonic or laser range 
finder, may be used. Alternatively or additionally, if the pattern on surface 21 is known (for 
example having been acquired in a pervious image), the scale of the currently acquired pattern 
may be used to determine contact with the paper. Alternatively or additionally, it, should be 
noted that when writing, the tip of the pen usually travels in a plane, unlike in spatial gesturing. 
An analysis of the acquired images may be used to determine that the pen is traveling on a 
plane and, hence, on a writing surface. Alternatively or additionally, it is noted that when 
holding a pen, axial movements are generally used to bring the pen to a writing surface, while 
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transverse movements are usually used to guide the pen over the surface. Movement with 
limited or no axial component may be assumed to be connected with a writing surface. 
Alternatively or additionally, it is noted that ink is generally dispensed only when ink dispenser 
30 is in contact with surface 21. Analysis of the acquired images may be used to determine if 
any additional ink was dispensed. 

In a preferred embodiment of the invention, a single pen 21 may be used for multiple 
modes of data input. Fig. 4 is a schematic representation of a data entry pad 70, in accordance 
with a preferred embodiment of the present invention. In a preferred embodiment of the 
invention, the pad is printed on a plain or a special paper sheet, for example using a laser 
printer and/or a photo-copier. Pad 70 may include a designated mouse pad portion 72, a 
designated writing portion 74 and/or one or more designated function key portions 76. 
Preferably, pen 20 operates in different modes over each of the portions. In mouse pad portion 
72, relative movement of pen 20 moves a cursor on a computer screen (not shown). Preferably, 
mouse pad 72 (and possibly other portions of entry pad 70) is coated with a material which 
resists ink from ink dispenser 30. Alternatively or additionally, mouse pad 72 is erasable. 
Alternatively or additionally, when mouse pad 72 is all marked up, a new data entry pad 70 is 
used. Alternatively or additionally, a disappearing ink is used. Mouse pad portion 72 may be 
automatically detected based on a background pattern printed thereon and identified by pen 20. 
Alternatively or additionally, a switch on pen 21 is used. Alternatively or additionally, a 
gesture (described below) is used to change modes in pen 20. 

When pen 20 is placed over a function key portion 76, pen 20 preferably generates a 
signal indicating that the function key was "pressed". Preferably a degree of hysteresis is 
provided so that a second depressing requires moving the pen a significant distance away. A 
particular function key may be identified based on OCR of the text printed in the key area. 
Alternatively or additionally, a special pattern is imprinted on the background of the key area. 
Alternatively or additionally, a one or two dimensional bar-code is imprinted and read by pen 
20. Alternatively or additionally, a control or a gesture is used, as for the mouse function 
described above. 

Writing portion 74 is preferably used for handwriting input, as described above. 

Figs. 5A-5F comprise a flowchart of a program for controlling a pen input device, in 
accordance with a preferred embodiment of the invention. It should be appreciated that this 
flowchart is only an example of one possible control structure for a pen input device, many 
other control structures will occur to a person skilled in the art. 
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Fig. 5A illustrates a main loop, in which depressing of two mouse button simulating 
switches (mousel, mouse2) is determined. In Figs. 5B and 5C, a signal is sent to a host 
computer responsive to pressing and/or releasing of the buttons. 

Fig. 5D illustrates the operation when camera 22 is operational, for. example responsive 
5 to a camera switch being activated. If imaged area 24 includes a pattern of mouse pad 72, a 
mouse advance process is activated. If imaged area 24 includes a pattern of a function key, the 
key is detected and then sent to the host computer keyboard driver. Otherwise, if imaged area 
24 includes a pattern of a writing portion 74, the pattern is used to assist in determining 
movement of pen 20. Preferably, no ink is released if any of the above patterns are identified, 
10 since none are required. Ink dispenser 30 is preferably electronically controlled to restrict flow 
of ink When not needed. If imaged area 24 does not include any of the above background 
patterns, ink dispenser 30 is preferably electronically controlled to allow ink to flow onto a 
writing surface. 

In Fig. 5D, handwriting recognition utilizes a "replica algorithm", described below, 
15 which identifies movements of the pen and/or whether a continuous line was created. In a 
preferred embodiment of the invention, especially when bit-mapped based OCR is used, an 
image is generated in a bit-map image memory, which image includes the contribution of the 
individual pen motions. Such an image is preferably filtered to remove any underlying grid. 
Alternatively or additionally, if ink dispenser 30 was electronically controlled not to dispense 
20 ink, the bit map is modified to include a line which would have been drawn if ink had been 
allowed to flow. Thus, the bit map may be generated by assessing the contributions of pen 
motions rather then by combining images. Removing an underlying pattern is thus greatly 
simplified or not required at all. Additionally or alternatively, removing markings generated by 
the pen itself is also facilitated, by knowledge of the pen dispensing ink and/or the pen 
25 movements. 

The image memory (which may be maintained for example on the pen and/or on an 
attached computer) is then analyzed to detect a character and/or a gesture^ If no character is 
identified, further pen movements may be required in order to recognize a character. 
Preferably, a temporal threshold is used to determine when no further movements should be 
30 accepted. If no character is identified, an error code may be sent to the host computer or a 
default character may be recognized. Alternatively another recognition library is checked. Once 
a character is recognized, its code is preferably sent to a keyboard driver of the host computer 
and the bit-map memory is preferably erased. 
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Fig. 5E illustrates a "replica algorithm". A new and a previous images are correlated 
and the correlation peak determined. Additionally, a determination is made to see if there is a 
contiguous drawn line from between an origin in the previous image and the origin in the new 
image. The movement and the existence of a line are preferably returned to a calling program. 

Fig. 5F illustrates a "mouse advance algorithm". A new and a previous image are 
correlated and the peak correlation determined. The amount shift required for the peak 
correlation is returned as a movement. 

Electronic ink control may be activated, as described above, by an underlying pattern. 
Alternatively or additionally, the activation may be controlled by an activation mode of the 
pen, for example, responsive to movement of the pen, such as a gesture and/or using a physical 
control on the pen (which is electrically coupled to ink dispenser 30). In a preferred 
embodiment of the invention, the electronic ink dispenser comprises a valve which controls the 
flow of ink. Alternatively or additionally, the electronic ink dispenser comprises an actuator 
which retracts an ink-dispensing tip relative to the tip of pen 20, so that the ink dispenser does 
not contact a writing surface and dispense ink. 

In a preferred embodiment of the invention, other types of data input are facilitated by 
pen 20. In one example, pen 20 may be used for scanning images, with the scanned areas 24 
being assembled as a mosaic, preferably using correlation. In another example, pen 20 is used 
to enter graphics, which may be stored as bit maps and/or as vectors. In another example, pen 
20 is used for filling out and/or amending forms. Preferably each field on the form has a 
different background pattern, so that they can be individually identified. Alternatively or 
additionally, each filed includes a bar-code and/or an alphanumeric code which can be acquired 
by camera 22. 

In a preferred embodiment of the invention, pen 20 may be used to enter gestures, 
which are movements of pen 20 which do not generally correspond to any identifiable 
character. Examples of gestures include editing marks (e.g., swap letters), selection of text, 
special commands to pen, for example to increase resolution or control ink flow and/or gestures 
to change the mode of operation of the pen (e.g., handwriting mode -> mouse mode). In a 
preferred embodiment of the invention, gestures are associated with a location relative to 
printed text, for example for editing of forms. 

As can be appreciated, various less or more complicated and/or automated logics may 
be defined for a pen input device, utilizing the myriad of available inputs: physical switches, 
imaged patterns, text and/or bar codes, gestures and/or movements and/or utilizing one or more 
of the myriad of available functional modes, such as handwriting recognition, scanning, 
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function key entry, pen control and/or mouse function. Additionally, pen 20 may provide one 
or more of a plurality of feedback functions, including, sound (from a host computer or the 
pen), display of input received at a host computer, a display on the pen itself and/or messages 
on the host computer. 

5 In a preferred embodiment of the invention, pen 20 is used to mark up a pre-printed 

sheet of paper, the content of which is preferably known to a host computer. If the content is 
not known, the paper may be scanned using pen 20. Various methods may be provided for 
absolute localization of pen 20 on the paper. In one example, the paper is pre-printed with a 
varying pattern which is unique at each point on the paper. In another example, such a pattern 

10 (or grid) is provided as a transparency. In another example, the printed matter is utilized as 
such a varying pattern, providing that area 24 is large enough to always include a uniquely 
printed section of paper. In some cases, especially if the imaged text is unique, it is not 
necessary to determine a position. Rather, the annotations performed by pen 20 are directly 
applied to the text, which text is determined based on its being unique. In another example, pen 

15 20 is moved along the paper from a landmark position, so that the position of pen 20 relative to 
the landmark position is known. In some of the above cases, text on the pre-printed paper is 
identified. In other cases, only a low-resolution image of the text needs to be used. In another 
example, a forest (of unconnected graph segments) of local annotations and/or images of 
sections of the paper are acquired. Gradually, individual localities on the paper are merged, as 

20 areas 24 which overlap two such localities are acquired. It should be appreciated that this type 
of merging of imaged localities may also be useful in handwriting recognition, if the pen 
moved more than the size of area 24 in one step. 

In a preferred embodiment of the invention, when a grid or other pre-printed pattern is 
provided for the writing surface, the pattern is preferably printed using an invisible and/or 

25 marginally visible ink, for example an ER ink, which is visible to camera 22. Alternatively or 
additionally, a fluorescent ink is used, which preferably emits light in response to light, such as 
UV light, from a light source on pen 20. 

In a preferred embodiment of the invention, pen 20 is calibrated prior to use. 
Calibration preferably includes setting the field of view of camera 20 and/or its focusing. In a 

30 preferred embodiment of the invention, when camera 22 is provided as a separate bracket 
mounting as shown in Fig. 1, for example, camera 22 is automatically calibrated using the tip 
of the pen as a point of reference. Alternatively or additionally, such automatic calibrations 
may utilize a grid or other patterned portion of a data entry pad. 
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In a preferred embodiment of the invention, a pen may be personalized for a particular 
user, language, target application program and/or style of use. The following items may be 
customized, for example: the language, writing direction, shape of characters, use of shorthand, 
preferred gestures and their meaning (e.g., application control, entering of non-printing 

5 characters, and/or assigning scripts commands and/or macros to gestures), writing angle, and/or 
rotation of the pen. In a preferred embodiment of the invention, a user can view such a 
personalization on a computer screen, so that he can keep track of them. In some embodiments 
of the invention, the pen will upload the gestures to the computer, to be displayed. 
Alternatively or additionally, different resolution mouse pads may be used for different people. 

10 In some cases, the same mouse pad is used to emulate different resolution steps. Alternatively, 
the pen identifies a step size on the mouse pad and uses this step size to select a mouse 
movement resolution and/or step. In a preferred embodiment of the invention, personalization 
information is stored in a database on the computer and/or on the pen. When the information is 
stored on the pen, a display and/or controls on the pen are used to view and/or modify the 

15 information. Alternatively or additionally, the maintenance of the information is via a computer 
connection, which preferably either directly controls the pen or downloads information and/or 
settings to it. 

It should be noted that pen 20 may be used, in some embodiments of the invention, to 
enter characters in non-Latin languages and/or pictogram-based languages, such as Arabic, 
20 Japanese and Chinese. An advantage of pen 20, in some embodiments thereof, over prior art 
devices, is that the combination of imaging and motion detection may be used to determine and 
track high resolution features of the writing, for example stroke width. 

In a preferred embodiment of the invention, especially when entering a gesture, the 
motion of the pen may be differently interpreted depending on what is already printed on the 
25 surface. In one example, a small "v" over a line of printed text will be interpreted as a gesture 
to insert the following text, while on a blank page it will be interpreted as a letter "V'\ In 
another example, the written markings may combine with existed pre-printed (or pre-written) 
marking to firm a single symbol, character and/or pattern to be recognized. 

Fig. 6 is a schematic illustration of a system configuration, in accordance with a 
30 preferred embodiment of the invention including, a computer 82, a pen input device 20 and/or a 
dedicated hardware 80 which preferably includes a DSP. The following three dimensions of the 
configuration may be identified: 

(a) Is required processing performed in pen 20, computer 82, using dedicated hardware 
80 (possibly installed in computer 82) or in a combination of the above. 
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(b) Are the connections between pen 20, computer 82 and dedicated hardware 80 wired, 
wireless or a combination. 

(c) Does pen 20 require real-time communication with computer 82 or are off-line 
operations supported and/or sufficient. 

5 Referring to dimension (a), in the current state of the art, the division of processing 

between pen 20, computer 82 and dedicated hardware 80 is a design question; any 
contemplated division may be implemented, at a cost, using current technology, with a possible 
speed tradeoff. In some embodiments, especially if a high powered CPU is available no 
dedicated hardware and/or processing on the pen is used at all. 

10 Referring to dimension (b), again, any contemplated interconnection scheme may be 

implemented, however, it is noted that processing and/or image acquisition require a significant 
amount of power, thus, it may be desirable for pen 20 to be connected by a wire to one of the 
other elements of the system. Alternatively or additionally, a quick-charge docking station may 
be provided, so that pen 20 is only required to store power for short periods of time, between 

15 docking instances, referring to dimension (c), it is noted that a main usage of pen 20 is as a 
real-time input device to computer 82. However, it should be noted that for marking up a pre- 
printed paper, for marking up or filling a form, for map navigating, for measuring a distance 
along a route on a map, for making measurements on an architectural drawing (or display) 
and/or for scanning text, images, graphics and/or handwriting, there is no need to communicate 

20 with a host computer. In addition, if feedback means, such as a display, are provided on the 
pen, the pen can form a complete computer system in itself. A pen display suitable for 
incorporation in a pen 20, in some preferred embodiments of the invention, is described for 
example in EP patent 0094867, the disclosure of which is incorporated herein by reference. 

Referring to dimension (c), in a preferred embodiment of the invention, pen 20 can 

25 operate in a stand alone mode and upload data to a computer when it is connected to one. the 
data may be partially analyzed by the pen. Alternatively the pen may utilize an online 
connection with the computer. Alternatively or additionally, the computer may be used to 
download updated programming, configuration files and/or personalization settings to the pen. 
Fig. 7 A and 7B are block diagrams of a wired pen input device in accordance with a 

30 preferred embodiment of the invention. Fig. 7A is a block diagram of a dedicated device 80 
suitable for a wired pen. Although an analog interface to the pen is shown, a digital or a hybrid 
interface may also be using in some preferred embodiments of the invention. Alternatively or 
additionally, the image is conveyed from the pen using a fiber-optic cable. Fig. 7B is a block 
diagram of the pen itself. Special note should be taken of the power supply management 
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blocks, which may be used to put the pen and/or dedicated hardware 80 in a power-saving 
hibernating state if the pen is motionless. In such a state, the image acquisition rate may also be 
reduced. Alternatively or additionally, pen 20 includes a movements sensor, so that image 
acquisition may be stopped. Alternatively or additionally, the pen includes an on/off switch. 
Alternatively or additionally, the pen includes a "hibernate switch". Alternatively or 
additionally, the pen includes a timer which reduces the electrical activity in the pen after a 
certain time has elapsed since a pen movement and/or a control activation. Other power saving 
techniques may also be applied. 

Fig. 8A and 8B are block diagrams of a wireless pen input device in accordance with a 
preferred embodiment of the invention. Fig. 8A is a block diagram of a dedicated device 80 
suitable for a wireless pen. Special note should be take of the connector for pen charging and 
the modem transceiver. The transmission may be optical, IR, RF, ultrasonic and/or other short- 
range transmission techniques known in the art. Fig. 8B is a block diagram of the pen itself. 
Special note should be take of the battery which may be required unless power is beamed to the 
pen, for example using micro-waves or magnetic field coupling. Preferably, power saving 
techniques, as described herein are also utilized in this embodiment. 

Various types of hardware interactions between the pen and computer 82 are possible. 
Some limited examples follow: on a simplest level, pen 20 may be connected (optionally via 
dedicated hardware 80) to computer 82 instead of a keyboard and/or a mouse. Alternatively or 
additionally, pen 20 may be connected to a serial port, preferably a USB (universal serial bus), 
to a parallel port, to a SCSI connector, to a Fire Wire port (1394 port) and/or as a network 
device. Additionally, dedicated hardware 80 may be implemented as a card in the host 
computer and the pen may be connected thereto. 

Various types of software interaction may be used. Some limited examples follow: on 
one end of a spectrum is the pen generating hardware signals of a mouse and/or keyboard, 
possibly emulating a standard device, such as a Microsoft® mouse. On the other end of the 
spectrum is allowing pen 20 to interact only with a dedicated program. In between possibilities 
include a driver for the pen replacing a keyboard and/or mouse driver and the driver for the pen 
providing input to an operating system and/or to the drivers, through a system provided data 
entry hook. 

It will be appreciated that the above described methods and apparatus of pen input may 
be varied in many ways, including, changing the order of steps, which steps are performed on- 
line and which steps are performed off-line. In addition various distributed and/or centralized 
configurations may be used to implement the above invention, preferably utilizing a variety of 
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software tools. In addition, a multiplicity of various features, both of methods and of devices 
have been described. It should be appreciated that different features may be combined in 
different ways. In particular, not all the features shown above in a particular embodiment are 
necessary in every similar preferred embodiment of the invention. Further, combinations of the 
above features are also considered to be within the scope of some preferred embodiments of the 
invention. Also within the scope of the invention are computer readable media on which 
software, for performing part or all of a preferred embodiment of the invention, are written. It 
should also be appreciated that many of the embodiments are described only as methods or 
only as apparatus. The scope of the invention also covers hardware and/or software adapted 
and/or designed and/or programmed to carry out the method type embodiments. In addition, the 
scope of the invention includes methods of using, constructing, calibrating and/or maintaining 
the apparatus described herein. 

It will be appreciated by a person skilled in the art that the present invention is not 
limited by what has thus far been described. Rather, the scope of the present invention is 
limited only by the following claims. As used in the following claims, the words "comprises' 1 , 
"comprising," "includes", "including" or their conjugates shall mean "including but not 
necessarily limited to". 
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CLAIMS 

1. A method for determining a motion of a tip on a surface having a pattern of distinct 
features marked thereon, comprising: 

imaging the pattern at least in the vicinity of the tip, using a two-dimensional camera 
coupled to the tip; 

comparing successive images acquired by the camera; and 
analyzing the images to determine the motion of the tip. 

2. A method according to claim 1, comprising forming at least a portion of said pattern by 
ink released from said tip. 

3. A method according to claim 1, wherein comparing comprises correlating at least parts 
of the images. 

4. A method according to claim 1 , wherein comparing comprises feature extraction and 
matching of said images. 

5. A method according to claim 1, comprising providing said pattern on the surface prior 
to the imaging, wherein said pattern includes a coded pattern for absolute positioning of the tip. 

6. A method according to claim 5, wherein the coded pattern comprises a plurality of 
areas, each with a different background pattern and wherein said camera has a field of view 
larger than an area of a single pattern. 

7. A method according to claim 6, wherein the plurality of areas comprises a plurality of 
square bar codes that form a checkerboard. 

8. A method according to claim 5, wherein the coded pattern is a coded grid, wherein each 
line of the grid is a specific combination of lines of varied thickness and spaces of different 
widths. 
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9. A method according to claim 8, wherein the coded grid has a likeness of a woven 
lattice. 



10. A method according to claim 1, wherein at least part of said pattern is invisible to a 
5 human eye. 

11. A method according to claim 10, wherein said pattern is reflective in the infrared 
spectrum. 

10 12. A method according to claim 1, wherein said pattern comprises a fluorescent material. 

13. A method according to claim 1, wherein said surface comprises a transparency having 
said pattern printed thereon. 

15 14. A method according to claim 1, wherein said surface comprises a plain paper with a 
pattern printed thereon. 

15. A method according to claim 1, wherein said pattern comprises a grain of the surface. 

20 16. A method according to claim 1, wherein determining the motion of the tip includes 
determining relative translation. 

17. A method according to claim 1, wherein determining the motion of the tip includes 
determining relative orientations. 

25 

18. A method according to claim 1, wherein determining the motion of the tip includes 
determining an absolute position. 

19. A method according to claim 16, wherein said position comprises a one-dimensional 
30 position. 

20. A method according to claim 16, wherein said position comprises a two-dimensional 
position. 
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21. A method according to claim 1, wherein determining the motion of the tip includes 
determining an absolute orientation. 

22. A method according to claim 17, wherein said orientation comprises a rotation angle of 
said tip around an axis perpendicular to the surface. 

23. A method according to claim 17, wherein said orientation comprises an angle between 
said tip and said surface. 

24. A method according to any of claims 1-23, wherein determining the motion comprises 
determining a velocity of the motion. 

25. A method according to any of claims 1-23, wherein determining the motion comprises 
determining a rate of change of the velocity of the motion. 

26. A method according to any of claims 1-23 comprising utilizing said detected motions to 
recognize handwriting. 

27. A method according to claim 26, wherein handwriting recognition comprises 
handwriting recognition utilizing motion vectors. 

28. A method according to claim 26, wherein handwriting recognition comprises utilizing 
both motion vectors and images of the surface for handwriting recognition. 

29. A method according to claim 26, wherein handwriting recognition comprises: 
generating an image utilizing said determined motion; and 

recognizing at least one symbol in said generated image. 

30. A method according to claim 26, wherein handwriting recognition comprises utilizing a 
user-dependent database to perform said handwriting recognition. 

31. A method according to any of claims 1-23, comprising utilizing said determined 
motions for graphical input. 
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32. A method according to any of claims 1-23, comprising utilizing said determined motion 
for controlling a cursor. 

33. A method according to claim 32, wherein controlling a cursor comprises emulating a 
mouse. . 

34. A method according to any of claims 1-23, comprising reconstructing a scanned image 
from said acquired images, utilizing said determined motions. 

35. A method according to claim 34, comprising analyzing said images to recognize special 
codes. 

36. A method according to any of claims 1-23, wherein said tip forms a part of a pen input 
device, and comprising: 

recognizing a particular pattern in at least one of said acquired images; and 
modifying an operational mode of said pen input device responsive to said recognized 
particular pattern. 

37. A method according to any of claims 1-23, wherein said tip forms a part of a pen input 
device, and comprising: 

receiving a voice input by said pen; and 

modifying an operational mode of said pen input device responsive to said voice input. 

38. A method according to any of claims 1-23, wherein said tip forms a part of a pen input 
device, and comprising: r 

recognizing an entry of a symbol utilizing said determined motions; and 
modifying an operational mode of said pen input device responsive -to said recognized 
symbol. 

39. A method according to any of claims 1-23, wherein said tip forms a part of a pen input 
device, and comprising: 

recognizing said determined motions as gestures; and 

modifying an operational mode of said pen input device responsive to said recognized 
gestures. 
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40. A method according to any of claims 1-23, wherein said tip forms a part of a pen input 
device, and comprising: 

detecting when the tip in contact with said writing surface; and 

modifying an operational mode of said pen input device responsive to said detection. 

41. A method according to claim 40, wherein said modifying comprises detecting motion of 
the tip responsive to said detected contact. 

42. A method according to claim 40, wherein said modifying comprises operating said 
camera responsive to said detected contact. 

43. A method according to claim 36, wherein modifying comprises selectively operating 
said pen input device in at least one of the following input modes: handwriting input, graphical 
input and pointer emulation. 

44. A method according to claim 43, wherein the input modes include a hypertext link 
selection mode, in which the pen may be selectively operated. 

45. A method according to claim 40, wherein modifying comprises controlling said camera. 

46. A method according to claim 36, wherein modifying comprises controlling the release 
of ink. 

47. A method of controlling ink flow in a pen, comprising: 

determining when said pen is in contact with a surface whose marking is allowed; and 
automatically electronically controlling a flow of ink, responsive to said determination. 

48. A method according to claim 47, wherein determining comprises determining if said 
pen is operating in a pointer emulation mode. 

49. A pen input device comprising: 
a tip; 
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a two dimensional camera, coupled to said tip, which images a surface at the vicinity of 
the tip; and 

circuitry for determining motion of said tip relative to said surface, by analysis of 
successively acquired images. 

50. A pen input device according to claim 49 comprising an electrical wire connection to a 
computer. 

51. A pen input device according to claim 49 comprising a wireless connection to a 
10 computer for data generated in the pen. 

52. A pen input device according to claim 50, wherein at least some of said circuitry is 
comprises in a dedicated processing unit. 

15 53. A pen input device according to claim 50, wherein at least some of said circuitry 
comprises software operable with on said computer. 

54. A pen input device according to any of claims 49-53, wherein said pen comprises a 
memory. 

20 

55. A pen input device according to claim 54, wherein said memory stores at least portions 
of said images. 

56. A pen input device according to claim 54, wherein said memory stores data responsive 
25 to said movement determination. 

57. A pen input device according to any of claim 49-53, wherein said pen comprises an 
internal power source. 

30 58. A pen input device according to any of claims 49-53, comprising a light source, coupled 
to the pen input device, to illuminate the surface. 

59. A pen input device according to any of claims 49-53 comprising a pressure transducer 
which determines if the tip is in contact with the surface. 
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60. A pen input device according to any of claims 49-53, wherein the pen input device is 
selectively operable in a plurality of input modes, including at least handwriting recognition 
and pointer emulation. 

61. A pen input device according to claim 60, wherein the input modes include pattern 
recognition. 

62. A pen input device according to claim 60, wherein the input modes include hypertext 
link selection. 

63. A pen input device according to claim 60, comprising an external software, wherein 
said mode is determined by the software. 

64. A pen input device according to claim 63, wherein said software controls the pen to 
operate in said different modes. 

65. A pen input device according to claim 63, wherein said software analyses signals from 
the pen, responsive to said mode. 

66. A pen input device according to claim 63, comprising a personal computer on which 
said software is executed. 

67. A pen input device according to any of claims 49-53, comprising at least one switch for 
controlling the operation of the pen. 

68. A pen input device according to any of claims 49-53, wherein said tip comprises an ink 
dispenser. 

69. A pen input device according to claim 68, comprising an ink flow control which 
selectively allows ink to be dispensed from said tip onto said surface. 

70. A pen input device according to any of claims 49-53, wherein said circuitry comprises a 
memory for reconstructing virtual pen marking from said determined movements. 
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71. A pen input device according to any of claims 49-53, wherein said circuitry recognizes 
handwriting patterns based on said determined movements. 

72. A pen input device according to any of claims 49-53, wherein said circuitry determine a 
three-dimensional position of said tip relative to said surface. 

73. A pen input device according to any of claims 49-53, wherein said circuitry determines 
a three-dimensional orientation of said tip relative to said surface. 

74. A pen input device according to any of claim 49-53, wherein said pen comprises a 
rechargeable power source. 

75. An implement mountable camera, comprising: 
a camera having a field of view; and 

a bracket for attaching said camera to a writing implement, such that said field of view 
images at least a vicinity of a tip of said elongate writing implement. 

76. A camera according to claim 75, comprising circuitry for analyzing images acquired by 
said camera to determine motion of said tip. 

77. A controllable pen having a pen tip, comprising: 
an ink dispenser; and 

an ink flow control which selectively blocks ink from being dispensed from said ink 
dispenser. 

78. A pen according to claim 77, wherein said ink flow controller comprises an electrical 
flow controller. 

79. A pen input device comprising: 
a tip; 

a two dimensional camera coupled to said tip and which images a surface at the vicinity 
of the tip; and 

a rechargeable power source for powering said camera. 
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80. A pen input device according to claim 79, comprising a connector to a computer, 
wherein said connector is adapted to hold said pen while not is use and wherein said connector 
comprises a data communication adapter for said computer and a power adapter to recharge 
5 said rechargeable power source. 
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